Image composition apparatus and method

ABSTRACT

Image composition apparatuses and methods are provided. The image composition method includes the steps of receiving a plurality of images having at least partial common scene, performing analysis on the plurality of images to determine at least one image feature of the plurality of images, determining at least two sub-regions by comparing the image feature between the plurality of images, determining at least two portions selected from at least two source images among the plurality of images, the at least two portions comprising the at least two sub-regions, generating a composite image by combining the at least two portions, and providing the composite image to a user.

This application claims the benefit of U.S. Provisional Application Ser.Nos. 61/759,440, 61/759,448, and 61/759, 444 filed on Feb. 1, 2013,which are hereby incorporated by reference in its entirety.

CROSS-REFERENCES TO RELATED APPLICATIONS

Not applicable.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to image composition apparatuses andmethods; more particularly, the present invention relates to imagecomposition apparatuses and methods that generate a composite image bycombining portions of different source images.

2. Descriptions of the Related Art

Digital image capture devices/units (e.g. digital camera mobile devices)have been evolved fast with better functionality and performances andare capable to capture multiple images in a relatively short time. Usermay perform burst mode shooting to capture a series of images eitherautomatically or manually. Normally, such series of images are capturedwith the same scene and object(s). Once such series of images areobtained, the series of images can be further processed to generatecomposite images with particular effect or composition.

Image composition is about generating a pleasant composite image withdesired effect by combining different parts of two or more images.Although lots of techniques and applications for composing images havebeen developed, most of them deal with images taken from different viewsand hence focus on accurately segmenting regions for composition andcolor blending at the boundaries of the composed regions. In themeantime, for those conventional image composition techniques, regionsto be composed usually can be easily identified.

In reality, images usually contain incomplete or noisy information aboutthe desired regions so that the regions to be composed are often poorlyidentified. That is, the initially obtained regions may have significantparts of unnecessary or undesired pixels but miss great parts of desiredpixels. In addition, the regions may have significant overlapping withcounterparts in other images. The imprecision of the identified regionsto be composed are not considered by conventional image compositiontechnique. Although this problem may be a proper task for the well-knowngraph-cut framework to work on, graph-cut may not be a practicalsolution for media production, as its memory-intensive nature preventsit from dealing with high resolution images on moderate computingplatform. Besides, such optimization approaches are often not intuitivefor further tuning and evaluation.

Consequently, there is still an urgent need for an efficient imagecomposition technique that can deal with the regions that can only bepoorly identified.

SUMMARY OF THE INVENTION

To solve the aforementioned problems, the present invention providesimage composition apparatuses and methods.

The image composition apparatus of the present invention comprises aninterface module, a confidence analysis module, a sub-region extractionmodule, a source determination module, and a sub-region growth module.The interface module is configured to receive a plurality of images,wherein the images are aligned and are defined by a plurality ofpositions and each of the images comprises a pixel at each of thepositions. The confidence analysis module is configured to calculate aconfidence value for each pixel of each image. The sub-region extractionmodule is configured to suppress a portion of the positions according tothe confidence values and determine a plurality of sub-regions for acomposite image according to the unsuppressed positions. The sourcedetermination module is configured to determine a source for each of thesub-regions, wherein each of the sources is one of the images. Thesub-region growth module is configured to grow the sub-regions until thecomposite image is filled by the sub-regions and fill each of thesub-regions by the pixels in a corresponding sub-region of thecorresponding source.

The image composition method of the present invention is executed by acomputing apparatus and comprises the steps of: (a) receiving aplurality of images, wherein the images are aligned and are defined by aplurality of positions, each of the images comprises a pixel at each ofthe positions, (b) calculating a confidence value for each pixel of eachimage, (c) suppressing a portion of the positions according to theconfidence values, (d) determining a plurality of sub-regions for acomposite image according to the unsuppressed positions, (e) determininga source for each of the sub-regions, wherein each of the sources is oneof the images, (f) growing the sub-regions until the composite image isfilled by the sub-regions, and (g) filling each of the sub-regions bythe pixels in a corresponding sub-region of the corresponding source.

Yet another image composition method of the present invention isexecuted by an image processing unit and comprises the steps of: (a)receiving a plurality of images, the plurality of images comprising atleast partial common scene, (b) performing analysis on the plurality ofimages to determine at least one image feature of the plurality ofimages, (c) determining at least two sub-regions by comparing the imagefeature between the plurality of images, (d) determining at least twoportions selected from at least two source images among the plurality ofimages, the at least two portions comprising the at least twosub-regions, (e) generating a composite image by combining the at leasttwo portions, and (f) providing the composite image to a user.

Briefly speaking, the present invention mainly calculates a confidencevalue for each pixel of each of the inputted images, suppresses aportion of the positions according to the confidence values, determinesa plurality of sub-regions for a composite image according to theunsuppressed positions, determines a source for each of the sub-regionsfrom the inputted images, grows the sub-regions until the compositeimage is filled by the sub-regions, and fills each of the sub-regions bythe pixels in a corresponding sub-region of the corresponding source.

According to the above description, it is learned that instead ofdeciding whether each position should be a part of a sub-region, thepresent invention suppresses positions that may not be covered by asub-region. Hence, the sub-regions determined by the present inventionhave strong tendencies on which source image they should come from (orwhich source image they should not come from) and meanwhile possibleoverlapping of multiple foreground objects is dealt with. Since thesub-regions are well determined, a pleasant composite image is thengenerated.

The detailed technology and preferred embodiments implemented for thesubject invention are described in the following paragraphs accompanyingthe appended drawings for people skilled in this field to wellappreciate the features of the claimed invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A, 1B, and 1C illustrate the concept of the present invention;

FIG. 2A illustrates a schematic view of the image composition apparatusof the first embodiment;

FIG. 2B illustrates the images received by the image compositionapparatus of the first embodiment;

FIG. 2C illustrates the determined sub-regions when the goal is tocreate the composite image having the moving objects contained in theinputted images;

FIG. 2D illustrates the composite image that has the moving objectscontained in the inputted images;

FIG. 2E illustrates the determined sub-regions when the goal is tocreate a clean plate;

FIG. 2F illustrates the composite image that is a clean plate;

FIG. 3 illustrates a flowchart of the image composition method of thesecond embodiment; and

FIG. 4 illustrates a flowchart of the image composition method of thethird embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following description, the present invention will be explainedwith reference to embodiments thereof. However, these embodiments arenot intended to limit the present invention to any specific environment,applications, or particular implementations described in theseembodiments. Therefore, description of these embodiments is only forpurpose of illustration rather than to limit the present invention. Itshould be appreciated that elements unrelated to the present inventionare omitted from depiction in the following embodiments and the attacheddrawings.

Please refer to FIGS. 1A, 1B, and 1C for the concept of the presentinvention. The objective of the present invention is to generate acomposite image 10 based on a plurality of images (e.g. the multipleimages 1, 2, 3, 4). The images 1, 2, 3, 4 are of a static scene and maycontain one or more moving objects, while the composite image 10 may bea clean plate of the images 1, 2, 3, 4 or an image having the movingobjects contained in the images 1, 2, 3, 4 along with a background. Toachieve the objective, the present invention mainly divides thecomposite image 10 into several sub-regions 10 a, 10 b, 10 c, 10 d asshown in FIG. 1A, decides the pixels of each of the sub-regions 10 a, 10b, 10 c, 10 d coming from which of the images 1, 2, 3, 4 (e.g. thesub-regions 10 a, 10 b, 10 c, 10 d are decided coming from the images 1,2, 3, 4 respectively as shown in FIG. 1B), and propagates the decisionto near-by pixels of the sub-regions 10 a, 10 b, 10 c, 10 d. When twogrowing sub-regions encounter, blending can be performed to combinepixel values from two source images. After the propagation, thecomposite image 10 is completely derived. Various embodimentsillustrating the present invention will be given below.

A first embodiment of the present invention is an image compositionapparatus 20 and a schematic view of which is illustrated in FIG. 2A.The image composition apparatus 20 comprises an interface module 21, aconfidence analysis module 23, a sub-region extraction module 25, asource determination module 27, and a sub-region growth module 29. Theconfidence analysis module 23 is electrically connected to the interfacemodule 21 and the sub-region extraction module 25, while the sourcedetermination module 27 is electrically connected to the sub-regionextraction module 25 and the sub-region growth module 29. The interfacemodule 21 may be any interface module that can receive and transmitdata.

Each of the confidence analysis module 23, the sub-region extractionmodule 25, the source determination module 27, and the sub-region growthmodule 29 may be realized by an individual processing unit or the like.In some other embodiments, the confidence analysis module 23, thesub-region extraction module 25, the source determination module 27, andthe sub-region growth module 29 may be integrated as an individualprocessing unit or the like. Yet in other embodiments of the invention,the confidence analysis module 23, the sub-region extraction module 25,the source determination module 27, and the sub-region growth module 29may be implemented as software programs or code sections that can beexecuted by one or more hardware processing units.

The interface module 21 receives a plurality of images 20 a, 20 b, 20 c,20 d, 20 e, 20 f of a static scene as shown in FIG. 2B. The images 20 a,20 b, 20 c, 20 d, 20 e, 20 f are defined by a plurality of positions; inother words, each of the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f has apixel at each of the positions. In addition, each pixel of each of theimages 20 a, 20 b, 20 c, 20 d, 20 e, 20 f has a pixel value. In thisembodiment, the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f are aligned sothat the corresponding points respect to part of the static scene wouldbe at the same position in each of the images 20 a, 20 b, 20 c, 20 d, 20e, 20 f.

Several remarks should be emphasized herein. First, the presentinvention does not limit the number of images that the interface module21 can receive, although six are received in this embodiment. Second, animage composition apparatus of the present invention may be equippedwith an additional image alignment module. For those embodiments, theinterface module 21 may receive images that have not been aligned andthe image alignment module will align the received images.

Next, the confidence analysis module 23 calculates a confidence valuefor each pixel of each of the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f.Each confidence value can be understood as how possible will the pixelappear on a composite image 28. The confidence analysis module 23calculates the confidence values based on different strategies dependingon the effect/content (e.g. a clean plate, multiple objects, or else)expected to be possessed by the composite image 28, which will bedescribed separately in the below description.

When the goal is to create a clean plate of a scene (i.e. the compositeimage 28 will be a clean plate), the confidence analysis module 23 mayfollow common background estimation techniques by calculating thestatistics of the pixel values of the images 20 a, 20 b, 20 c, 20 d, 20e, 20 f on the same position. Pixel values that appear more frequentlywould have higher confidence because creating a clean plate of a scenecould be done by composing multiple background regions reside in each ofthe images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f. Two concrete examples forcalculating confidence values are given below; however, it should benoted that the ways for calculating confidence values are not limited tothe following two examples.

In the first example, the confidence analysis module 23 performs thefollowing operations for each of the positions: (i) determining anappearance frequency for each of the pixels at the position according tothe pixel values of the pixels at the position and (ii) determining theconfidence values of the pixels at the position to be a value that ispositively correlated to the appearance frequencies of the pixels at theposition. Each of the confidence values is further normalized to a valuebetween 0 and 1 and then updated by subtracting the confidence valuefrom one. In the second example, the confidence analysis module 23performs the following operations for each of the positions: (i)calculating a reference value according to the pixel values of thepixels at the position, wherein the reference value may be a medianvalue, an average value, or the like, and (ii) deciding the confidencevalue of each of the pixels by calculating a difference of the pixelvalue and the reference value. Similarly, each of the confidence valuesis further normalized to a value between 0 and 1 and updated bysubtracting the confidence value from one. As illustrated in the twoexamples, the confidence values have to be updated by subtracting themfrom one when the goal is to create a clean plate of a scene. In thisway, the updated confidence values suggest the tendency of which imageshould not be selected as the source image of a sub-region in a laterstage performed by the sub-region extraction module 25. In thisembodiment, there will be six confidence values for a given positionbecause there are six images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f.

When the goal is to create the composite image 28 having the movingobjects contained in the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f alongwith a background, the confidence analysis module 23 calculates theconfidence values in an opposite manner. Two concrete examples forcalculating confidence values are given below; however, it should benoted that the ways for calculating confidence values are not limited tothe following two examples.

In the first example, the confidence analysis module 23 performs thefollowing operations for each of the positions: (i) determining anappearance frequency for each of the pixels at the position according tothe pixel values of the pixels at the position and (ii) determining theconfidence values of the pixels at the position to be a value that isnegatively correlated to the appearance frequencies of the pixels at theposition. In the second example, the confidence analysis module 23performs the following operation for each of the positions: calculatingthe confidence value of each of the pixels at the position bysubtracting a pixel value at the position of a reference image (e.g. aclean background image) from the pixel value. In both examples, each ofthe confidence values may be further normalized to a value between 0 and1.

Herein, it is emphasized that the following description related to thesub-region extraction module 25, the source determination module 27, andthe sub-region growth module 29 are the core of the present invention.The core of the present invention gives a great tolerance for theimperfection and illness created in the process of generating theconfidence values. Specifically, common problems that are likely to failconventional image composition techniques (e.g. imperfect imagealignment, imperfect background/foreground estimation, overlapping ofregions to be composited, or only rough user inputs could be obtained)would all be taken into consideration in the image composition method inthe present invention.

For convenience in the following description, it is assumed that theconfidence values 22 a, . . . , 22 b are derived after all the positionshave been processed. It is noted that the larger the confidence valueis, the more likely the pixel is at the background (or foreground). Eachof the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f may further form aconfidence map comprising the confidence values corresponding to all thepixels. Then, in the pixel suppression process of the composition stage(i.e. the operations performed by the sub-region extraction module 25),the confidence maps may be used to exclude undesired pixels within theinput images.

Next, the sub-region extraction module 25 determines a plurality ofsub-regions 24 a, . . . , 24 b for the composite image 28 based on theconfidence values 22 a, . . . , 22 b. It is noted that by determiningproper sub-regions 24 a, . . . , 24 b, the impact of the imperfectionand illness in the pre-processing stage (i.e. image alignment,confidence value calculations, etc.) could be reduced or recovered.

Briefly speaking, the sub-region extraction module 25 determines thesub-regions 24 a, . . . , 24 b that show strong tendencies on whichsource image they should come from (or which source image they shouldnot come from) and meanwhile deals with possible overlapping of multipleforeground objects. Instead of deciding whether each position should bepart of a sub-region, the sub-region extraction module 25 suppresses aportion of the positions that may not be covered by a sub-regionaccording to the confidence values 22 a, . . . , 22 b and determines aplurality of sub-regions 24 a, . . . , 24 b for the composite image 28according to the unsuppressed positions.

The sub-region extraction module 25 determines the sub-regions 24 a, . .. , 24 b for the composite image 28 based on different strategiesdepending on the effect/content (e.g. a clean plate, multiple objects,or else) expected to be possessed by the composite image 28 as well,which will be described separately in the below description.

When the goal is to create the composite image 28 having the movingobjects contained in the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f alongwith a background, the sub-region extraction module 25 determines thesub-regions 24 a, . . . , 24 b that show strong tendencies on whichsource image they should come from and deals with possible overlappingof multiple foreground objects. The sub-region extraction module 25 maysuppress positions according to the following three criteria (i.e. thefirst, second, and third/replacing criteria) individually or in anycombination.

Regarding the first criterion, the sub-region extraction module 25perform the following operation for each of the positions: suppressingthe position when the confidence values of the pixels of the images 20a, 20 b, 20 c, 20 d, 20 e, 20 f at the position are all smaller than afirst threshold. In other words, when the confidence values at aposition are all smaller than the first threshold, there is no strongtendency of which source image should come from. Hence, the position issuppressed.

Regarding the second criterion, positions that are overlapped by two ormore high-confidence foreground objects are dealt with. To be morespecific, the sub-region extraction module 25 performs the followingoperations for each of the positions: suppressing the position when theconfidence values of at least two pixels at the position are higher thana third threshold. The second criterion suppresses positions that haveno strong tendency of which source image that they should come from.

Regarding the third criterion, the sub-region extraction module 25perform the following operations for each of the positions: (i)determining the pixel that has the highest confidence value at theposition, (ii) locating a neighboring pixel of the pixel that has thehighest confidence value, and (iii) suppressing the position when theconfidence value of the neighboring pixel is lower than a secondthreshold. The objective of the third criteria is dividing thoseoverlapping object apart along the object edge of the one with a higherconfidence. In some other embodiments, the aforementioned thirdcriterion may be replaced by another criterion. Regarding this replacingcriterion, the sub-region extraction module 25 performs the followingoperations for each of the positions: (i) locating a first source imagethat comprises the pixel having the highest confidence value at theposition, the first source image is one of the images, (ii) locating asecond source image that comprises the pixel having the highestconfidence value at a neighboring position of the position, the secondsource image is one of the images, and (iii) suppressing the positionwhen the first source image and the second source image are differentand when the highest confidence value at the neighboring position islower than a second threshold.

The idea of using the above first and second criteria is to leave thepositions where do not show strong tendencies on which source image theyshould come from to be determined until the sub-region growing stepperformed by the sub-region growth module 29. In the meantime, the abovesecond and third criteria deal with the overlapping that occurs betweenregions of different images. The third criterion suffices to deal withsmall area of overlapping between regions as it tends to cut apart theoverlapped regions along the confidence edge of the region that containshigher confidence. But for large overlapping, often it is not able toobtain perfect cut-apart as it takes into consideration that theconfidence value computed in the pre-processing stage may be noisy. Thusan additional step of checking the suppressed results is performed.

When the goal is to create a clean plate (i.e. the composite image 28will be a clean plate), the sub-region extraction module 25 determinesthe sub-regions 24 a, . . . , 24 b that show strong tendencies on whichsource image they should not come from. This is because to create aclean plate, the same desired background part is likely to exist in twoor more input images. Thus the main concern would be identifyingforeground parts that should be avoided to put into the composed image28. From this perspective, the sub-regions needed for creating a cleanplate is actually pretty much like the sub-regions used for creating animage with multiple foreground objects. However, for creating a cleanplate, it is not necessary for overlapping foreground objects to beseparated, since in some cases, two or more foreground objects may becovered using a single piece of background parts. Hence, the sub-regionextraction module 25 may suppress positions according to theaforementioned first and third/replacing criteria individually or in anycombination.

After the sub-region extraction module 25 has dealt with all thepositions, the sub-regions 24 a, . . . , 24 b for the composite image 28are derived. It is noted that each of the sub-regions 24 a, . . . , 24 bis formed by a plurality of the connected positions of the unsuppressedpositions.

Next, the source determination module 27 determines a source for each ofthe sub-regions 24 a, . . . , 24 b. It is assumed that the sources 26 a,. . . , 26 b are determined for the sub-regions 24 a, . . . , 24 brespectively. Each of the sources is one of the images 20 a, 20 b, 20 c,20 d, 20 e, 20 f. To achieve a better performance, the sourcedetermination module 27 determines the source for each of thesub-regions 24 a, . . . , 24 b based on different strategies dependingon the effect/content (e.g. a clean plate, multiple objects, or else)expected to be possessed by the composite image 28 as well, which willbe described separately in the below description.

When the goal is to create the composite image 28 having the movingobjects contained in the images 20 a, 20 b, 20 c, 20 d, 20 e, 20 f alongwith a background, the source determination module 27 performs thefollowing operations for each of the sub-regions: (i) performing thefollowing operation for each of the images: calculating an averageconfidence value for the image by the confidence values in acorresponding sub-region of the image and (ii) determining the source ofthe sub-region as the image having the highest average confidence value.

When the goal is to create a clean plate (i.e. the composite image 28will be a clean background), the source determination module 27 performsthe following operations for each of the sub-regions: (i) performing thefollowing operation for each of the images: calculating an averageconfidence value for the image by the confidence values in acorresponding sub-region of the image and (ii) determining the source ofthe sub-region as the image having the lowest average confidence value.

Next, the sub-region growth module 29 grows the sub-regions 24 a, . . ., 24 b until the composite image 28 is filled by the sub-regions 24 a, .. . , 24 b and fills each of the sub-regions 24 a, . . . , 24 b by thepixels in a corresponding sub-region 24 a, . . . , 24 b of thecorresponding source. It is noted that when two growing sub-regionsencounter, blending can be performed to combine pixel values from twosource images.

Please refer to FIGS. 2B, 2C, and 2D. As shown in FIG. 2B, the images 20a, 20 b, 20 c, 20 d, 20 e, 20 f contains a person moving from oneposition to another across the scene. When the goal is to create thecomposite image 28 having the moving objects contained in the images 20a, 20 b, 20 c, 20 d, 20 e, 20 f along with a background, the sub-regions24 a, 24 b determined by the sub-region extraction module 25 based onthe aforementioned first, second, and third/replacing criteria are shownin FIG. 2C. As can be observed, the resulting sub-regions 24 a, . . . ,24 b cover the positions of the person in the some of the images 20 a,20 b, 20 c, 20 d, 20 e, 20 f, with small overlapping being cut apart andlarge overlapping being united. Thus, eventually only the person fromthe images 20 a, 20 b, 20 c, 20 e would present in the composite image28 as shown in FIG. 2D. Although noisy confidence values cause some ofthe background positions (e.g. trees), lines of the basketball court,and some of the desirable foreground positions (e.g. the leg and head ofthe person in the image 20 c) to be missed in FIG. 2C, these problemsare solved to generate a final pleasant composite result 28 as in FIG.2D.

Please refer to FIGS. 2B, 2E, and 2F. When the goal is to create thecomposite image 28 that is a clean plate, the sub-regions 24 a, . . . ,24 b determined by the sub-region extraction module 25 based on theaforementioned first and third criteria are shown in FIG. 2E and theresultant composite image 28 is shown in FIG. 2F.

Please note that in this embodiment, the background image and theforeground image can be generated from different source images. In someother embodiments, user inputs may be received from another interfacemodule (e.g. a touch screen). For those embodiments, the foregroundimage may be composed by input images designated by a user and thesource images are limited to those designated source images.

A second embodiment of the present invention is an image compositionmethod and a flowchart of which is illustrated in FIG. 3. The imagecomposition method is executed by a computing apparatus such as theimage composition apparatus 20 in the first embodiment.

At the beginning, step S301 is executed for receiving a plurality ofimages, wherein the images are aligned and are defined by a plurality ofpositions. In addition, each of the images comprises a pixel at each ofthe positions. Next, step S303 is executed for calculating a confidencevalue for each pixel of each image. To be more specific, the step S303may calculate the confidence values by the same ways as those describedin the first embodiment; hence, the details are not repeated herein.

Following that, step S305 is executed for suppressing a portion of thepositions according to the confidence values. To be more specific, thestep S305 may suppress the portion of the positions according to theconfidence values by the same ways as those described in the firstembodiment; hence, the details are not repeated herein. Next, step S307is executed for determining a plurality of sub-regions for a compositeimage according to the unsuppressed positions. To be more specific, thestep S307 may determine the sub-regions by the same ways as thosedescribed in the first embodiment; hence, the details are not repeatedherein.

Following that, step S309 is executed for determining a source for eachof the sub-regions, wherein each of the sources is one of the images.Particularly, the step S309 may determine the source for each of thesub-regions by the same ways as those described in the first embodiment;hence, the details are not repeated herein. Next, step S311 is executedfor growing the sub-regions until the composite image is filled by thesub-regions. Subsequently, step S313 is executed for filling each of thesub-regions by the pixels in a corresponding sub-region of thecorresponding source.

In addition to the aforesaid steps, the second embodiment can executeall the operations and functions set forth for the image compositionapparatus 20 in the first embodiment. How the second embodiment executesthese operations and functions will be readily appreciated by those ofordinary skill in the art based on the explanation of the firstembodiment, and thus will not be further described therein.

A third embodiment of the present invention is an image compositionmethod and a flowchart of which is illustrated in FIG. 4. The imagecomposition method is executed by a computing apparatus such as theimage composition apparatus 20 in the first embodiment.

Initially, the image composition method executes step S401 for receivinga plurality of images, wherein the plurality of images comprises atleast partial common scene. Next, step S403 is executed for performinganalysis on the plurality of images to determine at least one imagefeature of the plurality of images. In some other embodiments, the stepS403 may be achieved by a step of performing image alignment of theplurality of images (not shown) and a step of determining a plurality ofconfidence values of pixels within the plurality of aligned images (notshown). It is noted that the ways for calculating the confidence valuesare the same as those described in the first embodiment; hence, thedetails are not repeated again herein. Subsequently, step S405 isexecuted for determining at least two sub-regions by comparing the imagefeature between the plurality of images. In some embodiments of theinvention, step S405 may be achieved by the criterions described above.Next, step S407 is executed for determining at least two portionsselected from at least two source images among the plurality of images,wherein the at least two portions comprising the at least twosub-regions. It is noted that in some other embodiments, the imagecomposition method may execute another step for providing the at leasttwo source images to a user (not shown), another step for receiving auser input via a user interface module (not shown), and another step fordetermining the source images according to the user input (not shown).By these steps, the source images used in the step S407 can bedetermined

Following that, step S409 is executed for generating a composite imageby combining the at least two portions. In some embodiments, the stepS409 may generate the composite image by the way as described in thefirst embodiment; hence, the details are not repeated herein. Next, stepS411 is executed for providing the composite image to the user.

The image composition method of the second and third embodiments may beimplemented by a computer program which is stored in a non-transitorytangible machine-readable medium. When the computer program is loadedinto a computing apparatus, a plurality of codes comprised in thecomputer program will be executed by the computing apparatus toaccomplish all the steps described in the second and third embodiments.This non-transitory tangible machine-readable medium may be a read onlymemory (ROM), a flash memory, a floppy disk, a hard disk, a compact disk(CD), a mobile disk, a magnetic tape, a database accessible to networks,or any other storage media with the same function and well known tothose skilled in the art.

Briefly speaking, the present invention mainly calculates a confidencevalue for each pixel of each of the inputted images, suppresses aportion of the positions according to the confidence values, determinesa plurality of sub-regions for a composite image according to theunsuppressed positions, determines a source for each of the sub-regionsfrom the inputted images, grows the sub-regions until the compositeimage is filled by the sub-regions, and fills each of the sub-regions bythe pixels in a corresponding sub-region of the corresponding source.

According to the above description, it is learned that instead ofdeciding whether each position should be a part of a sub-region, thepresent invention suppresses positions that may not be covered by asub-region. Hence, the sub-regions determined by the present inventionhave strong tendencies on which source image they should come from (orwhich source image they should not come from) and meanwhile possibleoverlapping of multiple foreground objects may be dealt with as well.Since the sub-regions are well determined, a pleasant composite image isthen generated.

The above disclosure is related to the detailed technical contents andinventive features thereof. People skilled in this field may proceedwith a variety of modifications and replacements based on thedisclosures and suggestions of the invention as described withoutdeparting from the characteristics thereof. Nevertheless, although suchmodifications and replacements are not fully disclosed in the abovedescriptions, they have substantially been covered in the followingclaims as appended.

What is claimed is:
 1. An image composition apparatus, comprising: aninterface module, being configured to receive a plurality of images,wherein the images are aligned and are defined by a plurality ofpositions, each of the images comprises a pixel at each of thepositions; a confidence analysis module, configured to calculate aconfidence value for each pixel of each image; a sub-region extractionmodule, configured to suppress a portion of the positions according tothe confidence values and determine a plurality of sub-regions for acomposite image according to the unsuppressed positions; a sourcedetermination module, configured to determine a source for each of thesub-regions, wherein each of the sources is one of the images; and asub-region growth module, configured to grow the sub-regions until thecomposite image is filled by the sub-regions, and configured to filleach of the sub-regions by the pixels in a corresponding sub-region ofthe corresponding source.
 2. The image composition apparatus of claim 1,wherein each pixel has a pixel value, the confidence analysis moduleperforms the following operation for each of the positions: determiningan appearance frequency for each of the pixels at the position accordingto the pixel values of the pixels at the position; wherein theconfidence values of the pixels at the position and the appearancefrequencies of the pixels at the position are in a positive correlation.3. The image composition apparatus of claim 1, wherein each pixel has apixel value, the confidence analysis module performs the followingoperations for each of the positions: calculating a reference valueaccording to the pixel values of the pixels at the position, thereference value being one of a median value and an average value;wherein the confidence value of each of the pixels at the position iscalculated as the difference of the pixel value to the reference value.4. The image composition apparatus of claim 1, wherein each pixel has apixel value, the confidence analysis module performs the followingoperations for each of the positions: determining an appearancefrequency for each of the pixels at the position according to the pixelvalues of the pixels at the position; wherein the confidence values ofthe pixels at the position and the appearance frequencies of the pixelsat the position are in a negative correlation.
 5. The image compositionapparatus of claim 1, wherein each pixel has a pixel value, theconfidence value of each of the pixels at the position is calculated bysubtracting a pixel value at the position of a reference image from thepixel value.
 6. The image composition apparatus of claim 1, wherein thesub-region extraction module suppresses a portion of the positions byperforming the following operations for each of the positions:suppressing the position when the confidence values of the pixels at theposition are all smaller than a first threshold.
 7. The imagecomposition apparatus of claim 6, wherein the sub-region extractionmodule suppresses a portion of the positions by performing the followingoperations for each of the positions: determining the pixel that has thehighest confidence value at the position; locating a neighboring pixelof the pixel that has the highest confidence value; and suppressing theposition when the confidence value of the neighboring pixel is lowerthan a second threshold.
 8. The image composition apparatus of claim 6,wherein the sub-region extraction module suppresses a portion of thepositions by performing the following operations for each of thepositions: locating a first source image that comprises the pixel havingthe highest confidence value at the position, the first source image isone of the images; locating a second source image that comprises thepixel having the highest confidence value at a neighboring position ofthe position, the second source image is one of the images; suppressingthe position when the first source image and the second source image aredifferent and when the highest confidence value at the neighboringposition is lower than a second threshold.
 9. The image compositionapparatus of claim 7, wherein the sub-region extraction modulesuppresses a portion of the positions by performing the followingoperations for each of the positions: suppressing the position when theconfidence values of at least two pixels at the position are higher thana third threshold.
 10. The image composition apparatus of claim 1,wherein each of the sub-regions is formed by a plurality of connectedpositions of the unsuppressed positions.
 11. The image compositionapparatus of claim 1, wherein the source determination module performsthe following operations for each of the sub-regions: performing thefollowing operation for each of the images: calculating an averageconfidence value for the image by the confidence values in acorresponding sub-region of the image; and determining the source of thesub-region as the image having the lowest average confidence value. 12.The image composition apparatus of claim 1, wherein the sourcedetermination module performs the following operations for each of thesub-regions: performing the following operation for each of the images:calculating an average confidence value for the image by the confidencevalues in a corresponding sub-region of the image; and determining thesource of the sub-region as the image having the highest averageconfidence value.
 13. An image composition method, being executed by acomputing apparatus and comprising the steps of: (a) receiving aplurality of images, wherein the images are aligned and are defined by aplurality of positions, each of the images comprises a pixel at each ofthe positions; (b) calculating a confidence value for each pixel of eachimage; (c) suppressing a portion of the positions according to theconfidence values; (d) determining a plurality of sub-regions for acomposite image according to the unsuppressed positions; (e) determininga source for each of the sub-regions, wherein each of the sources is oneof the images; (f) growing the sub-regions until the composite image isfilled by the sub-regions; and (g) filling each of the sub-regions bythe pixels in a corresponding sub-region of the corresponding source.14. The image composition method of claim 13, wherein each pixel has apixel value and the image composition method further comprises thefollowing steps of: performing the following step for each of thepositions: determining an appearance frequency for each of the pixels atthe position according to the pixel values of the pixels at theposition, wherein the confidence values of the pixels at the positionand the appearance frequencies of the pixels at the position are in apositive correlation.
 15. The image composition method of claim 13,wherein each pixel has a pixel value and the image composition methodfurther comprises the following steps of: performing the following stepfor each of the positions: calculating a reference value according tothe pixel values of the pixels at the position, the reference valuebeing one of a median value and an average value; wherein the confidencevalue of each of the pixels at the position is calculated as thedifference of the pixel value to the reference value.
 16. The imagecomposition method of claim 13, wherein each pixel has a pixel value andthe image composition method further comprises the following steps of:performing the following step for each of the positions: determining anappearance frequency for each of the pixels at the position according tothe pixel values of the pixels at the position; wherein the confidencevalues of the pixels at the position and the appearance frequencies ofthe pixels at the position are in a negative correlation.
 17. The imagecomposition method of claim 13, wherein each pixel has a pixel value andthe confidence value of each of the pixels at the position is calculatedby subtracting a pixel value at the position of a reference image fromthe pixel value.
 18. The image composition method of claim 13, whereinthe step (c) comprises the following step of: performing the followingoperations for each of the positions: suppressing the position when theconfidence values of the pixels at the position are all smaller than afirst threshold.
 19. The image composition method of claim 18, whereinthe step (c) comprises the following step of: performing the followingoperations for each of the positions: determining the pixel that has thehighest confidence value at the position; locating a neighboring pixelof the pixel that has the highest confidence value; and suppressing theposition when the confidence value of the neighboring pixel is lowerthan a second threshold.
 20. The image composition method of claim 18,wherein the step (c) comprises the following step of: performing thefollowing operations for each of the positions: locating a first sourceimage that comprises the pixel having the highest confidence value atthe position, the first source image is one of the images; locating asecond source image that comprises the pixel having the highestconfidence value at a neighboring position of the position, the secondsource image is one of the images; suppressing the position when thefirst source image and the second source image are different and whenthe highest confidence value at the neighboring position is lower than asecond threshold.
 21. The image composition method of claim 19, whereinthe step (c) comprises the following step of: performing the followingoperations for each of the positions: suppressing the position when theconfidence values of at least two pixels at the position are higher thana third threshold.
 22. The image composition method of claim 13, whereineach of the sub-regions is formed by a plurality of connected positionsof the unsuppressed positions.
 23. The image composition method of claim13, further comprising the following steps of: performing the followingoperation for each of the images; calculating an average confidencevalue for the image by the confidence values in a correspondingsub-region of the image; and determining the source of the sub-region asthe image having the lowest average confidence value.
 24. The imagecomposition method of claim 13, further comprising the following stepsof: performing the following operation for each of the images;calculating an average confidence value for the image by the confidencevalues in a corresponding sub-region of the image; and determining thesource of the sub-region as the image having the highest averageconfidence value.
 25. An image composition method executed by an imageprocessing unit, comprising the steps of: receiving a plurality ofimages, the plurality of images comprising at least partial commonscene; performing analysis on the plurality of images to determine atleast one image feature of the plurality of images; determining at leasttwo sub-regions by comparing the image feature between the plurality ofimages; determining at least two portions selected from at least twosource images among the plurality of images, the at least two portionscomprising the at least two sub-regions; generating a composite image bycombining the at least two portions; and providing the composite imageto a user.
 26. The image composition method of claim 25, furthercomprising the steps of: receiving a user input via a user interfacemodule; and determining the source images according to the user input.27. The image composition method of claim 25, further comprising thesteps of: providing the at least two source images to the user.
 28. Theimage composition method of claim 25, wherein the step of performinganalysis on the images comprises the steps of: performing imagealignment of the plurality of images; and determining a plurality ofconfidence values of pixels within the plurality of aligned images.